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Section  l.  Program  Management  Plan 

1-L.  Review  the  program  PMD,  AFSC  Form  56,  ana  ESD  Form  211  to  determine  program 
specific  direction. 

1-2.  heview  AFSCR/AFLCR  800-2,  Management  of  Multi-Service  Systems,  Program1’.,  ana 
Projects  for  Tri-Service  Programs. 

1-J.  Review  AFR  800-2,  Acquisition  Program  Management  with  emphasis  on  tne 
attached  DOD  Directives. 

1-4.  Review  AFSCP  800-3,  A  Guide  for  Program  Management.  Particular  attention 
should  be  payed  to  Paragraph  3-7,  Chapter  23  -  Managing  Computer  Resources. 

1-5.  Prepare  tne  software  portion  of  the  PMP  so  that  it  provides  planning  for 
acquisition  management  and  management  visibility  of  project  status.  Insure 
coverages  of  the  following  areas,  when  applicable  (Ref  AFR  800-14,  Vol  II,  para 
3-7): 


a.  Operational  and  support  concepts  for  computer  programs  based  upon 
studies,  analyses,  experience,  command  inputs.  Organic  support  decisions  must  be 
based  on  AFR  26-12. 


b.  Identification  of  allocated  computer  resource  technical  and  managerial 
expertise . 

c.  Tne  system  engineering  approach  used  to  change  operational  requirements 
to  computer  resources. 


d.  Tne  pros  and  coos  of  hardwired  vs  programmable  computers,  including 
design  risk,  system  integrity  aod  life  cycle  cost. 


e.  Standardization  and  commonality  consideration  used  in  determining  system 
hardware  ard  software  requirements. 


f.  Computer  program  aod  data  rights  requirements. 

g.  A  master,  computer  resource,  development  schedule  showing  interrelation 
to  the  t,ota  l  system  acquisition  life  cycle. 

h.  Identification  of  required  intersystem  interfaces. 

i.  Growth  ana  spare  processing  capacity  requirements. 

j.  Requirements  for  acquisition  and  support  of  documentation. 

k.  Software  support  facility  requirements ;  simulation,  integration  and  otner 
applicable  facilities. 

l.  Configuration  management  concepts. 

m.  Criteria  for  the  transfer  of  program  management  responsibility  and 
system/equipment  turnover. 
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n.  Preparation  oi  tne  computer  Resources  Integrated  Support  Plan,  (CRISP). 

i-b.  insure  provisions  nave  been  made  to  develop  a  CRISP  (AFR  800-14,  Vol  II, 
Paragrapns  3-7,  ana  3-8). 

I-/.  Coordinate  draft  PMp  witn  program  office  and  all  applicable  outside  agencies 
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Section  2 .  Interlace  Control.  Working  Group 

2-1.  Review  AFK  800-14,  Volume  II,  Para  6-5. c  1'or  compliance  requirements. 

2-2.  Review  MIL-STD-483>  Appendix  II,  Para  20.6  tor  ICWG  compliance 
requirements. 

2-1.  Insure  tnat  the  ICWG  identities  and  controls  all  software  interfaces 
wnicn  cross  contract  boundaries  (e.g.,  external  to  the  given  contract  or  are 
externaL  to  this  system). 

2-4.  Participate  in  ICWG  meetings  covering  software  interfaces. 

2-5.  Assist  in  preparing  Interface  Control  Documents  (ICDs)  for  external 
software  interfaces. 

2-6.  Insure  that  the  contractor  designs  according  to  the  ICDs. 
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Section  j.  Computer  Hesourccs  integratea  Support  Planning 

j-l.  Heview  AFR  300-14,  Vol  II,  paras  3-8  and  3-10,  and  Chapter  10  for 
compliance  requirements. 

3-0.  Heview  AFLCH  800-21  for  informational  purposes  only. 

3-3-  Heview  AFR  06-1*1  for  Maintenance  Concepts. 

3-4.  Heview  AFH  800-8  for  Integrated  Logistics  Support  Planning  Concepts. 

3-5.  Heview  AFR  800-4  for  Program  Management  Responsibility  Transition 
Agreements. 

3-0.  Heview  AFR  800-19  for  Sy stem/Eouipment  Turnover  Agreement  Concepts. 

3-7.  Review  ESD  Guidebooks  ESD-TF-77-327  "Software  Acquisition  Management  - 
Software  Maintenance",  ESD-TR-77- i 30  "Software  Acquisition  Management  - 
Software  Development  and  Maintenance  Facilities"  and  ESD-TR-75-85  "AF  Guide 
for  Monitoriog  and  Reporting  Software  Development  Status". 

3-8.  Review  the  following  Program  Management  Documents  l when  they  first 
become  avaiLabLe). 

(a)  Program  Management  Directives  (PMD) 

(b)  Program  Management  Plan  (PMP) 

(c)  Request  for  Proposals  (RFP) 

(d)  Integrated  Logistic  Support  Plan  (ILSP) 

(el  Computer  Program  Development  Plan  (CPDP) 

(f)  Operational  Suppor t/Coof iguration  Management  Plan  (OS/CMP) 

(g)  Interface  Control  Documentation  (ICD)  (if  applicable) 

j-9.  Review  Figure  1-3-1,  Computer  Resources  Integrated  Support  Plan  Outline. 

3-t0.  Review  Figure  1-3-2,  Operational  Suppor t/Configuration  Management  Plan 
Out  line . 

3—11.  Review  Figure  1-3-3.  Computer  Resources  Integrated  Support  Plan 
Check  list . 

3-12.  Review  Figure  I-3-4,  Model  Computer  Resources  Working  Group  Charter. 
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Figure  1-3-1.  Computer  Resources  Integrated  Support  Han 
(Reference  AFLCR  800-21,  Attachment  7) 

1.0  Introduction. 

1.1  OVERVIEW.  (Give  the  purpose  of  the  CRISP  and  Identify  tne 
system/subsystem  it  addresses.  Include  a  brief  program  summary  along  with  the 
structure  of  CRISP,  i.e.,  number  of  volumes  and  subjects.) 

1.2  APPLICABILITY.  (State  any  pertinent  background  not  included  iri 
Paragraph  1.1.  Document  the  scope  of  the  CRISP  and  the  authority  (i.e.,  PMD, 

AFR  800-14,  etc.  ) ) 

1.3  REFERENCES.  (This  paragraph  may  include  abbreviations 'acronyms, 
glossary,  and  list  of  applicable  documents.  Basically  relate  the  CHISP  to 
other  CRISP  interfaces.) 

1.4  SYSTEM  DESCRIPTION.  (A  brief  description  of  this  weapon  system 
and/or  subsystem  and  present  status  of  the  system.  Not  a  detailed 
description. ) 

1.5  PROCESSOR (S )  AND  SOFTWARE  DESCRIPTION.  (Processor  identification 

and  a  description  of  the  software  (firmware)  associated  with  each 
system/subsystem.  Microprocessor  applications  should  be  identified  as  | 

software  intensive  or  hardware  intensive.  Firmware  should  be  classified  as 
software  intensive  (SWIF)  or  hardware  intensive  (HWIF).  A  block  diagraph 

snould  be  Included  to  provide  graphic  representation  of  the  system.) 

2.0  Management  Approach. 

2.1  MANAGEMENT  FOCAL  POINTS.  (This  section  should  contain  information 
on  CRWG  members,  organizations  involved,  offices  of  primary  responsibility 
(OPR)  and  their  responsibilities,  organizational  structure,  and  interface 
description  for  pre-  and  post-PMHT.  An  organizational  chart  should  be  * 

included. ) 


2.2  SUPPORT  CONCEPT.  (Details  of  the  support  should  include 
plans/procedures  to  establish  and  operate  the  support  facility  with  reference 
to  the  management  impacts.  Phase  charts  for  implementation  may  be  included. 
Identification  of  funding  requirements  should  be  documented  for  all  phases  of 
life  cycle  support.  Emphasis  snould  be  given  to  the  tasks  AFLC  performs  in 
support  of  development  before  PMRT  for  SPO  budgeting;  for  instance,  those 
tasks  performed  by  the  industrially  funded  Software  Support  Centers. ) 

2.3  SYSTEM/SUBSYSTEM  TURNOVER.  (This  section  details  the 
plans/procedures  for  operational  and  support  system  turnover.  It  gives 
procedures/plans  for  the  support  of  computer  programs  during  turnover. ) 

2.4  PROGRAM  MANAGEMENT  RESPONSIBILITY  TRANSFER  (PMRT).  (Paragraph  gives 
procedures/plans  for  operational  system  before  and  at  PMKT.  Include  the 
procedures/plans  for  support  system  operations  pre-  and  post-PMHT.  Separate 
PMRT  plans  may  be  referenced  where  appropriate,  and  will  become  a  part  of  this 
plan  to  the  extent  at  which  they  apply  to  computer  resources.  ) 
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2.5  SOFTWARE  CHANGES.  (This  paragraph  details  metnodoiogy  and  time 
constraints  for  reprogramming  actions  on  "ne  operational  programs  (l.e.,  block 
cnanges,  emergency  changes.)  informat  Lon  snould  include  pre-  and  post-PMRT. ) 

>.o  ML  ■  ie  I  CATIONS.  (Identify  procedures  for  modifications  to  the  system 
according  to  AFR  57-4.) 

c.,7  uEF ICIENCY  REPORTlNu.  (Identify  metnod  for  reporting  deficiencies 
in  computer  programs  and  procedures  to  correct  these  deficiencies.) 

3.  Conf igura tion  Management. 

3.1  GENERAL.  (Identify  basic  concepts  for  maintaining  configuration 
control  of  the  computer  resources.  Include  references  to  applicable  documents 
as  appropriate.  Also  reference  the  appendix  for  designating  the  CPCI  listing.) 

3.2  CONFIGURATION  CONTROL  RESPONSIBILITIES.  (This  section  should  detail 
tne  change  control  authority,  organizational  responsibilities,  and 
interaction/interface  between  acquiring,  using,  and  supporting  commands.  The 
information  should  cover  pre-  and  post-PMRT. ) 

3.3  CHANGE  CONTROL.  (Identify  the  plans  and  procedures  for 
recommending,  approving  and  processing  changes  to  the  computer  programs. 

These  cnanges  may  be  software  only,  software/hardware,  and  routine  vs 
emergency  change  requirements.  These  plans  and  procedures  should  cover  pre- 
and  post-PMRT. ) 

3.1)  STATUS  ACCOUNTING.  (Identify  the  CPCI  configuration  baseline  and 
procedures  for  accounting  for  implementation  of  the  change(s).) 

3.5  COMPUTER  PROGRAM  CONFIGURATION  ITEM  IDENTIFICATION  (CPIN). 
(Identification  of  tne  computer  programs  as  configured  items  and  procedures 
for  assigning  Computer  Program  Identification  Number.  Reference  CPIN 
Compendium  80-1  and  AFR  800-21,  Chapter  11.) 

4.0  Documentation. 

4.1  DATA  RIGHTS.  (This  paragraph  should  address  data  rights  and 
licensing  agreements  for  all  software.) 

4.2  OPERATIONAL  SYSTEM  DOCUMENTATION.  (Identify  the  documentation 
required  to  support  the  operational  system  (i.e.,  organic,  contractor)  which 
will  assume  timely  support  of  all  involved  computer  programs  according  to  the 
support  concept.  Include  need  dates  and  transfer  methods.' 

4.3  DOCUMENTATION  CONTROL  P..AHS /PROCEDURES.  (This  section  should  cover 
the  documentation  control  plans, procedures  for  controlling  and  updating 
documentation.  Also  address  stor  ,ge  in  i"s/procedures  for  documentation.) 

4.4  DOCUMENTATION  IDENTIFICATION  ( CPiN). 
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5.0  Personnel  ana  Training. 

5-1  PERSONNEL.  ILdentify  personnel  ana  specialty  requirements  for 
managing  and  supporting  the  computer  resources  involved.  Tnis  section  should 
also  identity  tne  contractor  resources  required  for  interim  contractor  support 
and  funding  responsibility.) 

5.2  TRAINING.  (Tnis  section  should  identify  the  training  required 
(formal  and  informal)  to  support  the  computer  resources  involved  and  ensure* 
successful  operation  and  management  of  the  system.) 

o.O  Support  Equipment/Sof twar e  and  Facility  Requirements. 

6.1  EQUIPMENT  REQUIREMENTS.  (Identify  supporting  command  equipment 
required  to  support  the  operational  software  programs  following  PMRT.  The 
concept  for  acquisition,  integration,  and  operation  of  the  support  equipment 
and  plans  for  verification  and  validation  of  the  support  equipment  shoula  be 
identified.  ) 

6.2  SOFTWARE/FIRMWARE.  (Identify  the  software/f irmware  programs 
required;  method  for  acquisition,  integration,  and  operation:  plans  for 
ver lfication,  validation,  and  engineering  analysis  of  the  operational 
software;  and  related  mission  equipment  interfaces.) 

6.3  SUPPORT  SOFTWARE  (GENERAL).  (Describe  support  software  (i.e., 
computer,  translators,  scientific  subroutines,  media  to  media  conversion 
programs,  etc.,)  required  and  identify  associated  documentation  if  not 
identified  in  Section  6.0.) 

6.6  FACILITIES.  (Describe  tne  requirements  for  physical  housing  of 
support  equipment  and  plans  to  establish  housing.) 

6.5  MAINTENANCE  OF  COMPUTER  RESOURCES  EQUIPMENT.  (Identify  a  plan  to 
maintain  the  equipment  to  include  funding  responsibilities.) 

7.0  Te3t  Support. 

/.I  OPERATIONAL  TEST  AND  EVALUATION.  (Identify  field  requirements , 
adaption  for  operational  personnel,  special  support  equipment,  special  support 
procedures,  interfacing  agencies,  and  any  special  maintenance  requirements. ) 

8.0  Verification  and  Validation  (VAV). 

8.1  OPERATIONAL  SOFTWARE.  (Identify  verification  and  validation  and 
acceptance  testing  requirements  for  computer  programs  involved.  Include  plan 
and  requirement  for  independent  VAV  of  the  operational  software  and  any 
interlacing  agencies.  Responsibilities  and  procedures  for  VAV  prior  to  and 
following  PMRT.) 

8.2  SUPPORT  SOFTWARE.  (Identify  verification  and  validation  and 
acceptance  testing  requirements  of  support  software.  Plans  and  requirements 
for  independent  VAV  of  support  software.  Separate  test  plans  and  procedures 
developed  for  the  purpose  may  be  referenced.  Responsibilities  and  procedures 
tor  VAV  prior  to  and  following  PMRT.) 
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9-0  Security,  (.identity  any  special  security  handling  procedures  and  the 
impact  of  the  security  procedures  on  operational  support.) 

lo.O  Security  Assistance.  (Identify  the  sale  or  possible  sale  of  the  system 
to  a  foreign  country,  the  equipment/software  that  is  oot  releasible  to  foreign 
countries,  and  the  support  concept/responsibilities.) 
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(Heierence  AFLCH  800-21,  Attachment  b) 

1.0  INTRODUCTION.  (A  genera L  overview  of  the  applicable  system,  commands 
supporting,  implementing  and  using  the  system,  and  areas  of  emphasis  in  th^ 
procedures.  ) 

1.1  PURPOSE.  (Discusses  what  the  procedures  will  establish  in  regards 
to  support  operations,  responsibilities  and  relationships  of  involved 
organizations  and  which  are  to  be  used  from  system  turnover  and  post-PMRT. ) 

1.2  SCOPE.  (Identifies  specific  command  organizations,  configuration 
management  requirements ,  ana  functional  responsibilities  affected  by  this 

p lan. ) 

l.j  APPLICABILITY.  (Define  the  relationship  of  this  plan  to  controlling 
documents  (AFRs)  and  other  management  documents.  Applicable  Memorandums  of 
Agreement  will  be  superseded  by  the  appropriate  0/S  CMP.) 

1.4  DEFINITIONS/ ABBREVIATIONS. 

I . b  APPLICABLE  DOCUMENTS. 

2.0  MANAGEMENT  FOCAL  POINTS.  (These  sections  shall  explicitly  delineate 
commands  and  their  affected  organizations '  authority,  responsibilites, 
interrelationship,  conf iguration  management  requirements,  and  operation 
management  requirements . ) 

2.1  CONTROL  BOARDS,  PANELS,  AND  GROUPS.  (Single  control  points  for  the 
ECS  and  each  of  its  CPCIs  will  be  addressed  in  terms  of: 

1.  Control  authority  will  be  defined,  delineated,  and  control  point 
interfaces  explicitly  defined. 

2.  Responsibilites  will  be  delineated  to  comply  with  AFR  800-14 
Chapter  6  and  the  applicable  involved  Command’s  management  plans. 

j.  Control  boards/panels  will  be  explicity  defined  in  terms  of 
membership,  authority,  responsibility,  and  interface  relationships.) 

2.2  ORGANIZATION  RESPONSIBILITIES.  ( Suppor t/User/Iroplemeoting 
organizations  will  oe  defined  in  terms  of: 

1.  System  and  CPCI  requirements  and  responsibilities. 

2.  Requirements  for  system  and  CPCI  support. 

8.  Participation  in  verif ication/ va lidation  test  planning, 
accomplishment,  and  evaluation. 

4.  Participation  in  system  and  CPCI  change  process. 

8.  Participation  in  Quality  Assurance  activities. 


9 


i it-, ,  Volume  i 
program  Management 


i.  February  ’.98 


3-0  CONFiuUHATi.CN  MANAGEMENT.  (Tnese  paragraphs  will  explicitly  define  AFP 
800-14  Cnapter  6  requirements  for  classifying  configuration  changes, 
identiiying  conf iguration  items,  ana  management  phases.) 

3.1  CHANGE/DISCREPANCY  CLASSIFICATIONS. 

3. P  COMPUTER  PROGRAM  CONFIGURATION  ITEMS. 

4.0  DEFICIENCY  REPORT/CHANGE  PROCEDURES.  (These  paragraphs  will  detail 
emergency,  urgent,  and  routine  discrepancy  and  cnange  processing  requirements 
tor  support  activities  prior  to  PMRT  and  post-PMRT.  These  procedures  will  be 
unique  to  tms  0/S  CMP.  Existing  AFR's  will  be  referenced  with  0/S  CMP 
detailing  the  necessary  actions  and  Interfaces  required  to  comply  with  those 
AFHs . 

4. L  REPORTING  REQUIREMENTS.  (Specifically  address  missioo  data  changes, 
nardwar e-software  changes,  hardware-only  changes,  and  software-only  changes.) 

4.P  C00RDINAT iNG .  (Processing  and  coordination  requirements  and 
respon3ioi l ities  tor  deficiency  reports  and  changes.) 

4.3  CONTROL  BOARD/PANEL  PROCESSING.  (Cnange  processing  requirements  for 
each  cootroi  boar d/pane  i/group . ) 

4.4  CHANGE/ DISCREPANCY  PROCESSING.  (Discrepancy  fault 
aetection/isolation  requirements  and  responsibilities.  CPCI  requirements  and 
responsioi lilies  relative  to: 

1.  CPIN  assignment. 

2.  Cnange  requirement  establishment,  disposition,  workloading. 

3.  Change  implementation  validation. 

4.  Error  recurrence  cootroi. 

5.  Interface  Test  Adapters  (ITA)  requirements  and  responsibilites 

tor  design,  development,  documentation,  and  delivery.) 

4.8  CORRECTIVE  ACTION  ACCOMPLISHMENT.  (Change/Discrepancy  corrective 
action  accomplishment  requirements  and  responsibiliteis  by  category  of 
discrepancy  and  change.: 

4.6  QUALITY  ASSURANCE.  (Identify  procedures  to  ensure  testing, 
documentation,  deficiency  reporting,  validation,  and  certification  are 
suuonip  1 1  sneu .  ) 

4.7  REPOSITORY.  (Library  repository  responsibilities  for  CPCI  build, 
impound,  and  reproduct  ion . ) 

4.8  DOCUMENTATI ON.  (Engineering  and  User  Data  update  responsibilities 
and  procedures  tor  corrective  actions/changes.) 
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4.9  SYSTEM  INTEGRATION.  (CPCI  aoa  System  integration  requirements  and 
responsibilities. ) 

4.10  DISTRIBUTION.  (CPCI  media  and  documentation  release  and 
distribution  requirements  and  responsibilities.  Specifically  identify  release 
points,  receiving  points,  and  CPCI  Office  of  Primary  Responsibility.) 

4.11  SAFETY  REQUIREMENTS. 

6*0  STATUS  ACCOUNTING.  (These  paragraphs  shall  define  requirements  for 
determining  the  status  of  the  approved  configuration  baseline,  proposed 
changes,  and  tne  status  of  changes  approved  for  implementation.) 

5.1  REPORTING  REQUIREMENTS. 

6.2  STATUS  ACCOUNTING  SYSTEMS. 

6.0  COMMUNICATIONS  REQUIREMENTS .  (These  paragraphs  will  identify  each 
specific  interface  of  involved  organizations  by  individual,  organization, 
phone,  and  address.  Organizations  requiring  identification  are  control 
boards,  media  distribution  point,  documentation  distribution  point,  and 
media/documeotatioo  receiving  point.  Requirements  for  electronic 
communication  shall  be  defined.) 

7.0  SECURITY.  (Tnese  paragraphs  shall  define  expansion  of  AFR  800-14 
requirements.  If  none  is  required,  these  paragraphs  should  so  state.) 

APPENDICES  (The  different  coof iguration  management  plans  applicable  to  the 
ECS’s  CPCI’ a  will  be  attached  as  individual  appendices. ) 
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Figure  x-3-j.  Computer  nesources  Integrated  Support  Plan  Checklist 

1.  Has  tne  purpose  ot  tne  CRISP  Deen  defined  along  with  its  scope  and 
autnor ity? 

?.  Have  tne  system  acquisition  pnases  been  defined  (including 
product ion/ deployment )? 


Have  the  appLicaDle  regulations  pertaining  to  the  information  in  the  CRISP 
been  referenced? 

4.  Does  a  scnedule  exist  in  the  CRISP  which  reflects  the  program  events  and 
their  milestones  during  the  developmental  and  production  phase  of  the  system 
acqui sition? 

5.  Does  tne  aDove  schedule  include  Program  Management  Responsibility  Transfer 
(PMRT)? 


b.  Has  a  brie!  description  of  the  weapon  system  and/or  subsystem  and  present 
status  of  tne  system  been  written? 

7.  Have  r esponsibi 1 ities  for  composite  system  integrity  been  defined  to 
inc  lude : 


a) 

o) 

c) 

d ) 

e) 


computer 
computer 
computer 
computer 
uti  lizat 


storage  utilization 

program  operating  time  and  priorities 
program  interface  techniques 
program  baseline  integrity 
ion  of  computer  modules  and  peripherals 


B.  ilas  tne  CRIMP  identified  the  computer  programs  required  to  support  the 
Computer  equipment  and  other  computer  programs  including  acquisition 
re3ponsibi l ities? 

<j.  Has  the  CRIMP  identified  tne  computer  equipment  and  devices  required  to 
facilitate  computer  program  changes,  including  acquisition  responsibilities? 


U).  Have  tne  Computer  Resources  Wortcing  Group  members  been  identified?  Have 
the  imp lementing ,  supporting  and  using  commands  been  identified? 

11.  Have  the  offices  o!  primary  responsibility  and  management  focal  points  for 
support  of  computer  resources  and  the  channels  of  communication  among 
organizations  been  identified  and  defined? 

Have  organizations  responsible  for  funding,  scheduling,  and  system 
integration  been  identified? 
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13-  Have  other  appropriate  working  groups  been  defined?  For  example: 

a.  Transfer  and  Turnover  (TTWG) 

b.  Interface  Control  Working  Group  (ICWG) 

c.  Any  required  joint  service  working  groups  (i.e,  joint  service  working 
group  and  joint  test  working  group) 

d.  Test  Planniog  Working  Group  (TPWG) 

14.  Have  details  of  the  support  concept  been  explained  including 
plans/procedures  to  establish  and  operate  the  support  facility  with  reference 
to  management  impact? 

lb.  Has  the  CRISP  defined  the  plan(s)  to  establish  and  operate  necessary 
support  facilities?  NOTE :  Common  and  existing  facilities  will  be  used 
whenever  practicable. 

lb.  Have  the  provisions  for  the  transfer  of  program  management  responsibility 
beeo  defined? 

17-  Have  provisions  for  the  turnover  of  the  system/equipment  been  made? 

L8.  Has  the  turnover  and  transfer  (clause)  of  a  system  been  provided? 

ly.  Has  the  methodology  for  software  changes  and  time  constraints  for 
reprogrammiog  actions  been  explained  in  detail? 

20.  Have  deficiency  reporting  procedures  been  defined? 

21.  Have  provisions  been  made  for  planning  for  configuration  management  of 
computer  programs  including  the  assignment  of  configuration  control 
responsibi lities  during  the  deployment  phase?  NOTE :  This  should  also  include 
the  Development  and  Transition  and  Turnover  Phases.  This  planning  should 
reflect  the  operational  and  support  concepts  for  the  system. 

22.  Has  an  organization  been  designated  to  chair  the  Configuration  Control 
Board  (CCB)  and  the  Computer  Program  Configuration  Sub-Board  (CPCSB)  during 
each  system  acquisition  phase? 

2 1 .  Have  data  rights  included  along  with  licensing  agreements  been  explained 
for  the  software? 

24.  Have  provisions  been  made  by  the  program  office  to  obtain  Computer  Program 
Identification  Numbers  (CPINs)? 

2b.  Ha3  tne  documentat ion  required  to  support  each  type  of  computer  program 
been  identified? 

26.  Are  documentation  resources  specified  in  the  Contract  Data  Requirements 
List? 
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2[.  Has  a  list  of  data  items  been  included  as  an  appendix?  Has  tailoring  been 
addressed? 

,•».  Have  tne  training  requirements  for  the  user  and  A FLC  been  identified?  Has 
follow-on  training  been  identified?  Is  the  training  equipment  needed  to 
support  tne  training  provided  by  the  implementing  command  or  ATC? 

29 .  Have  qualified  personnel  been  identified,  by  organization,  to  support  the 
computer  equipment  and  computer  program  at  PMRT? 

30-  Has  the  computer  resources  working  group  considered  Independent 
Verification  aoo  Validation?  Has  the  recommendation  to  the  program  manager 
been  documented? 

jl.  Have  provisions  been  made  for  the  verification  ana  validation  process  of 
computer  programs  throughout  the  system  life  cycle? 

i2 .  Is  the  draft  CRISP  fully  updated  and  coordinated? 

33-  Is  the  final  prepared  for  coordination/signature? 
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Figure  1-3-4.  Model  Computer  Resources  Working  Group  Charter 
A- l  GENERAL 

Tne  (Program  Name)  Computer  Resources  Working  Group  (CRWG)  is  established  and 
chartered  in  accordance  with  AFR  800-14,  this  regulation  being  used  as 
guidance  tor  the  (Program  Name)  Program.  Tne  CRWG  will  be  established  prior 
to  the  beginning  of  the  Full  Scale  Development  (FSD)  Phase.  It  will  remain  in 
existence  under  the  chairmanship  of  a  designated  repr esentati ve  of  the  program 
director  until  Program  Management  Responsibility  Transfer  ( PMRT) .  Following 
PMRT,  the  chairmanship  of  the  CRWG  will  be  amended  as  mutually  agreed  upon  by 
the  supporting  and  using  organizations. 

A -2  PURPOSE 

The  purpose  of  tne  CRWG  is  to  prepare  and  update  the  Computer  Resources 
Integrated  Support  Plan  (CRISP)  for  the  (Program  Name).  In  so  doing,  the  CRWG 
will  identity,  plan,  initiate,  and  coordinate  activities  necessary  to 
accomplish  these  objectives.  The  CRWG  will  address  those  support  activities 
which  require  the  interaction  of  multiple  organizations.  The  CRWG  will  serve 
as  a  forum  for  the  supporting  and  using  organizations  to  discuss,  communicate 
and  agree  on  recommendations  for  the  resolution  of  problems  in  the  support  of 
computer  resources.  It  will  provide  insight  into  support  activities  and  their 
ef fectiveness. 


A-3  RESPONSIBILITIES 


The  CRWG  will  nave  the  following  responsibilities  in  support  of  (Program  Name) 
computer  resources : 

a.  Develop,  update  and  maintain  the  Computer  Resources  Integrated  Support 
Plan  (CRISP)  in  accordance  with  AFR  800-14. 

d.  Assist  the  program  office  in  evaluating  FSD  activities  in  connection 
with  computer  resources. 

c.  Identify  those  elements  which  should  be  included  in  the  Turnover  and 
PMRT  Agreements,  and  insure  that  those  elements  are  addressed  in  the 
Agreements. 

A-4  CRWG  MEMBERSHIP 

At  it3  inceptioo  and  at  all  times  prior  to  PMRT,  the  CRWG  will  be  chaired  by  a 
designated  representati ve  of  the  (Program  Name)  Program  Director.  The  group 
will  be  made  up  of  members  who  will  represent  their  organization's  positions 
and  advisors,  who  will  only  comment  on  such  positions.  The  group  will  have 
representation  from  the  following  agencies,  boards  and  groups: 

Members: 

System  Program  Office 

ADCOM 

AAC 

CF 
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ATC 

AFCS 

AFLC 

Advisors:  (as  applicable): 

Computer  Program  Configuration  Sub-Board  (CPCSB) 

Configuration  Control  Board  (CCB) 

Human  Factors  Board  (HFB) 

Test  Planning  Working  Group  (TPWG) 

Turnover  and  Transfer  Working  Group  (TTWG) 

System  Safety  Group 
Contractors : 

Following  contract  award,  tne  prime  contractor  may  be  asked  to  participate  in 

selected  CRWG  meetings. 

A-5  CRWG  ADMINISTRATION 

Tne  CRWG  will  be  administered  in  accordance  with  the  following  procedure: 

a.  Minutes  of  CRWG  meetings  will  be  prepared  and  distributed  within  15 
days  following  each  meeting. 

b.  When  practicable,  CRWG  members  and  advisors  will  be  notified  of 
meeting  dates,  locations,  and  proposed  agenda  at  least  30  days  prior 
to  the  meeting. 

c.  CRWG  action  items  will  be  tracked  by  the  CRWG  chairman  and  their 
status  will  be  reported  at  each  CRWG  meeting. 

d.  Agenda  items  may  be  submitted  to  the  program  director  by  CRWG  members 
15  days  prior  to  each  scheduled  meeting. 

A-b  PROGRAM  SUMMARY 

The  following  major  periods  of  the  (Program  Name)  Program  are  applicable  to 

the  planning  of  computer  resources  support: 

a.  Validation  Phase 

b.  Full  Scale  Development  Phase 

c.  Production/Deployment  Phase 

d.  Operational  Phase 
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Section  jK  Turnover  and  Transfer 

4-1.  He  view  AFR  800-L4,  Volume  II,  Cnapter  9,  and  AFF  800-4  for  T&T 
r equirementa . 

4-2.  Heview  updatea  DD  Form  250  to  identify  all  deficiencies  and  exceptions 
that  are  still  to  be  delivered  or  corrected. 

4-8.  Heview  ESD-TH-7Y-326 «  Validation  and  Cer i ’ f ication ,  Para  8.2  to  insure 
all  requirements  in  AFF  800-14,  AFR  800-19  and  AFSCR/AFLCR  80-17  have  been  met. 

4-4.  Work  with  Transfer  Working  Group  (TWG)  to: 

a.  Insure  that  tne  CRISP  and  CRWG  agreements  are  incorporated  in  the 
system  Turnover  and  Transfer  Agreements. 

b.  Insure  that  all  software  support  requirements  are  adequately  reflected 
in  these  agreements. 

c.  Assist  in  identifying  computer  resource  deficiencies  and  exceptions  at 
time  of  turnover. 
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Section  3.  Early  Warning  Guidance  Indicators 

Tnis  section  is  intended  to  provide  Early  Warning  Guidance  ( EWG )  to 
Program  Offices.  It  is  not  intended  to  be  all  encompassing.  Each  unique 
application  snoulo  he  taiLoreo.  Some  questions  may  not  be  appropriate  where 
as  others  may  be  insufficient  in  detail.  This  document  will  be  constantly 
undergoing  revisions  as  experience  and  insight  is  gained  in  the  role  of  EWG. 


There  are  three  checklists  tailored  to  different  acquistion  phases: 

I-5-l,  Conceptual  pnase  EWG  Indicators;  1-5-2,  Validation  Pnase  EWG 
Indicators;  and  1-5-3,  Development  Phase  EWG  Indicators.  Each  phase  is  split 
into  seven  areas  of  coocern:  Software  Quality  Assurance,  Configuration 
Management,  Testing,  Turnover  &  Transfer,  Documentation,  Engineering  &  Program 
Management,  and  Contract  Hequirements.  Since  each  phase  covers  such  a  broad 
period  of  time,  all  the  questions  should  be  reviewed  throughout  the  phase  and 
asked  at  an  appropriate  time.  In  some  cases  is  will  be  necessary  to  ask  the 
question  several  times  because  of  an  insufficient  answer  or  due  to  a  need  for 
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Figure  1-5-1.  Conceptual  Phase  Early  Warning  Guidance  Indicator; 
Software  Quality  Assurance  ( SQA ) 

Have  government  personnel  been  allocated  to  perform  the  software  quality 
control  function? 

Has  tnere  been  an  assessment  for  future  needs  of  SQA  personnel? 

Is  tnere  an  SQA  Plan  for  government  personnel  to  follow? 

Do  organizational  cnarts  indicate  the  SQA  personnel  will  be  separate  from  the 
developers  and  report  to  the  same  management  level? 

Configuration  Management  (CM) 

Have  government  personnel  been  assigned  to  carry  out  the  CM  practices  and 
procedures?  (AFR  800-14,  Vol  II) 

Testing 

Has  the  TEMP  been  written  in  accordance  with  AFR  80-14  and  AFSC  Sup  1? 
Turnover  and  Transfer  (T&T) 

Has  the  Logistics  Support  Plan  been  written?  (AFR  800-8) 

Have  supporting  parties  been  notified  of  their  responsibility?  (AFR  800-14 
AFSC  Sup  l,  para  6) 

Documentation 

Does  the  initial  system  specification  reflect  the  operational  requirement 
accurately  and  fully?  Has  this  been  verified  by  the  Using  Command? 

Does  it  address  all  tne  topics  of  model  System  Spec  Para  3.3.8? 

Have  the  functions  of  the  system  been  allocated  to  Configuration  Items  (CIs)? 
(MIL-STD-483  &  490) 

Contract  Requirement 

Will  the  appropriate  standards  and  specifications  be  called  out  in  the 
contract,  i.e.,  will  MIL-STDe  483.  490,  1521A,  ....  and  MIL-S-52779A  be 
invoKed?  Have  they  been  tailored  to  this  specific  procurement? 

Is  tnere  a  CM  Plan,  SQA  Plan,  CPDP? 

Will  deliverables  provide  proof  of  adherence  to  every  standard? 

Have  approved  High  Order  Language  (H0L)  and  Top  Down  Structured  Programming 
(TDSP)  been  properly  addressed? 

Are  the  deLivery  r equir ement j ,  including  media  and  form,  specified  In  the 
contract?  (AFR  800-14,  Vol  II,  Para  8-5) 
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Mgt-Program  &  Engineering  Mgt 

Has  tne  Program  Manager  coordinated  with  AFLC ,  AFCMD,  ATC  and  any  other 
necessary  focal  points  to  cover  support  requirements  of  the  system?  i.e.,  are 
a l L  necessary  channels  of  communication  open  and  are  the  required  parties 
notified  of  their  responsibility  for  supporting  the  system?  (AFR  800-14,  AFSC 
Sup  l,  Para  6). 
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Figure  1-5-2.  Validation  Phase  Early  Warning  Guidance  Indicators 
T&T  (Turnover  &  Transfer) 


Have  provisions  been  made  to  ensure  incor poration  of  CRISP  provisions  in 
Turnover  and  Program  Management  Responsibility  Transfer  (PMRT)  documents? 
(AFR  800-14,  Vol  II,  Para  9-2) 

Have  Turnover  and  PMRT  been  considered  in  PMP  and  CRISP  planning?  (AFR 
800-14,  Vol  II,  Para  9-2) 

Have  using  and  supporting  commands  identified  computer  resources  and 
associated  documentation  necessary  to  support  the  system?  (AFR  800-14,  Vol 
II,  Para  10-3) 

Oo  the  plans  provide  for  easy  transition  to  the  using  command  and  their 
procedures? 

Documentation 


Are  the  documentation  requirements  periodically  evaluated?  (AFR  800-14,  Vol 
II,  Para  7-3d) 

Have  agreements  from  using  and  supporting  commands  been  obtained  for 
modifications  or  deletions  to  data  defined?  (AFR  800-14,  Vol  II,  Para  7-4) 

Where  is  the  software  program  documented  and  is  such  documentation  available 
for  review?  Are  you  getting  the  Version  Description  Document  (VDD)? 

Is  there  a  software  development  schedule? 

Do  the  SQA  procedures  provide  for  the  review  of  software  documentation? 
(MIL-3-52779A  3-2.4) 

Are  there  procedures  to  review  the  software  documents  throughout  the  software 
life  cycle? 

Do  procedures  exist  that  provide  for  in-depth  review  of  the  design 
specification,  interface  specification,  and  data  base  design  specification  to 
assure  efficiency,  effectiveness,  reliability,  testability  and  adherence  to 
standards? 

Do  the  SQA  procedures  provide  for  the  timely  monitoring  of  resources  in  the 
area  of  processing  time,  storage  and  input/output  channels  to  ensure  the 
proper  realization  of  specified  reserves? 

Mftt 

Ha3  a  Computer  Resources  Integrated  Support  Plan  been  drafted  and  has  the 
Computer  Resource  Working  Group  been  established?  (AFR  800-14,  Vol  II,  Para 
3-10) 

Are  the  Computer  Resources  Integrated  Support  Plan  and  Computer  Resources 
Working  Group  current?  (AFR  800-14,  Vol  II,  Para  3-10) 
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Has  tne  Computer  Program  Development  Plan  been  used  to  evaluate  contractor 
approaches?  (APR  800-14,  Vo l  II,  Para  3-9) 

li  tne  Computer  Program  Development  Plan  incorporated  into  the  contract?  ( AFR 
800-14,  Vo l  II,  Para  8-2a) 

Have  system  studies  been  planned/accomplished  to  select  design  approaches  and 
integrate  computers  aod  computer  programs  into  the  total  system?  (AFR  800-14, 
Vol  II,  Para  4-5a) 

Have  system  performance  and  interface  requirements  been  allocated  to  computer 
equipment  and  computer  program  configuration  items?  (AFR  800-14,  Vol  I,  Para 

3®(2)> 

Are  periodic  meetiogs  with  contractor,  using  aod  supporting  commands  required 
to  assess  the  progress  of  the  CPCIs?  (AFR  800-14,  Vol  II,  Para  4-10) 

Contract  Requirement 

Do  solicitation  documents  include  explicit  statements  establishing  Air  Force 
rights  to  CPCIs  aod  associated  documentation?  (AFR  800-14,  Vol  I  &  AFSC  Supp 

I,  Para  31) 

Does  tne  work  breakdown  structure  and  cost  reporting  system  separately 
identify  computer  programs  and  related  tasks?  (AFR  800-14,  Vol  II,  Para  8-3a) 

Are  alL  computer  programs,  equipment,  and  documents  necessary  to  support  and 
modify  the  operational  programs  identified  and  deliverable?  (AFR  800-14,  Vol 

II ,  Paras  3-8  &  8-3c) 

Are  appropriate  safeguards  included  in  vendor  contracts  to  support  the 
Government's  requirements  with  respect  to  system  configuration  level?  (AFR 
800-14,  Vol  II,  Para  8-6) 

Does  the  contract  have  a  requirement  for  a  Software  Quality  Assurance 
Program?  ( MIL-S-52779A ) 

Do  you  have  a  software  quality  assurance  program  which  assures  compliance  with 
the  requirements  of  the  contract?  (MIL-S-52779A ,  paras  1.2  4  3.2.4) 

Are  there  procedures  for  design  reviews  to  ensure  that  provisions  of  the 
contract  are  met?  (MIL-S-52779A  para  3.2.6) 

t)o  procedures  exist  that  ensure  the  performance  specification  satisfies  the 
requirements  of  the  SOW  and  other  governing  specifications?  (MIL-S-52779A 
para  j.2.6) 

Do  tne  SQA  procedures  assure  the  cooduct  of  design  walk-throughs? 
(MIL-STD-1679) 

Do  the  procedures  provide  for  the  adnerence  to  standards?  ( MIL-S-52779A  para 
1-3) 

Does  the  SQA  Program  contain  a  procedure  to  identify  conflicting 
requirements?  (MIL-S-52779A  para  3.2.4) 
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Testing 

Does  the  test  program  include  provisions  to  stress  test  computer  equipment  and 
computer  program  configuration  items?  (AFR  800-14,  Vo  l  II,  Para  0-1) 

Has  the  Preliminary  Qualification  Test  process  been  evaluated  and  applied:' 

(AFR  800-14,  Vo  1  II,  Para  5-3a(2).(a)) 

Do  provisions  exist  tor  integration  and  test  of  CPCls  into  the  system?  (AFR 
800-14,  Vo l  II,  Para  5-3D) 

Do  the  CPCI  development  specifications  identify  test  approaches  for  qualifying 
the  CPCI  and  contractor  requirements  for  GFE  and  support  requirements?  (AFR 
800-14,  Vo  1  II,  Para  8-5e) 

Does  the  system  level  DT&E  plan  identify  overall  planning  for  CPCI  testing  and 
does  it  contain  provisions  for  installation  and  checkout  of  CPCTs  into 
follow-on  insta  l  lations?  (AFR  800-14,  Vol  II,  Para  8-8f(lUl?)) 

Are  separate  test  procedures  prepared  for  each  qualification  test?  (AFR 
800-14,  Vol  II,  Para  5-5g) 

Do  qua l i t ication  tests  properly  cover  requirements  for  test  and  evaluation  of 
CPCIs?  (AFR  800-14,  Vol  II,  Para  5-5g) 

Are  the  results  of  all  CPCI  tests  documented?  (AFR  800-14,  Vol  II,  Para  F-Sh) 

Are  the  procedures  for  the  review  of  test  activities  included  in  the  SO A 
Program?  ( M1L-S-D277VA  3-2.8) 

Do  the  SQA  procedures  provide  for  the  analysis  of  software  requirements  to 
determine  testability?  ( MI L-S-82779 A  3.2.8) 

Do  SoA  procedures  provide  for  tne  review  of  test  procedures  for  compliance 
with  appropriate  standards,  performance  requirements  and  other  contractual 
requirements?  (MT(.-o-‘.277?A  3 . . : . 8  ) 

Do  i>wA  procedures  provide  tor  tne  monitoring  ot  tests  and  certification  that 
test  results  are  the  actual  findlt  of  the  test?  ( MI L-S-S2779A  2.2.8) 

Do  SQA  procedures  provide  for  tne  review  and  certification  of  test  reports? 
(MIL-S-92779A  3-2-8) 

Are  the  test  validation  procedures  adequate?  ( MI  L-,S-C,,-'77QA  3-2.8) 

Do  SQA  procedures  assure  that  test  related  documentation  Is  accurately 
maintained  to  allow  repeatabi  i  Ity  of  tests?  ( M11.-S-V779A  3.2.8) 

Is  the  organisation  responsible  for  testing  activities  identified? 

Are  the  various  testing  cycles  identified? 

Does  tne  SQA  program  identify  the  software  testing  activities? 
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Do  S(JA  procedures  provide  for  tne  review  or'  test  plans  for  compliance  witn 
appropriate  standards,  performance  requirements  and  otner  contractual 
requirements?  (MIL-S-52779A  3.2.8) 

CM 

Are  computer  equipment  and  computer  programs  covered  during  tne  conduct  of 
system  design  reviews,  audits  and  management  assessments.  (AFR  800-14,  Vol  I, 
Para  3m. (10)) 

Are  formal  technical  reviews  required  for  all  computer  equipment  and  computer 
program  configuration  items?  (AFR  800-14,  Vol  II,  Para  4-9) 

Is  a  successful  PDH  required  prior  to  detail  design? 

Is  a  successful  Critical  Design  Review  required  for  each  CPCI  prior  to 
proceeding  into  coding?  (AFR  800-14,  Vol  II,  Para  4-9c) 

Are  all  deliverable  computer  programs  identified  and  managed  as  CPCIs?  (AFR 
800-14,  Vol  I,  Para  3m(8);  Vol  II,  Paras  8-4b  &  8-2a(l)) 

Are  system  and  configuration  item  level  interfaces  specified  and  controlled? 
(AP«  800-14,  Vol  II,  Paras  8-5a  &  8-5c) 

Is  change  control  applied  to  CPCIs?  (AFR  800-14,  Vol  II,  para  6-8a) 

Are  configuration  audits  required/accomplished  for  all  CPCIs?  (AFR  800-14, 

Vol  II,  Para  5-8) 

Are  computer  firmware  aod  micr opr ocessor* /microcomputers  treated  as 
configuration  items  aod  documented  accordingly?  (AFR  800-14,  Vol  I,  AFSC  Sup 
1,  Para  3-®(8)  ATT  I,  Para  3;  MIL-S-52779A  1.1) 

Is  tnere  a  Configuration  Management  plan?  (MIL-STD-483) 

Is  the  Conf iguration  Management  Plan  adequate? 

Are  the  designated  personnel  qualified  to  perform  Configuration  Management 
Audits? 

Are  the  SQA  procedures  for  auditing  Configuration  Management  available  and 
documented?  (MIL-S-52779A  3-2.7,  MIL-STD-483) 

Do  QA  procedures  exist  to  review  aod  approve  changes  to  configured  software; 
i.e. ,  SCP,  ECP,  etc.? 

Do  procedures  exist  tor  the  timely  reporting  of  corrective  action  for  the 
aoa lysis  of  problems  and  deficiency  reports  to  determine  trends  extent  and 
causes?  ( MIL-S-52779A  3-?. 9) 

Do  procedures  exist  for  the  timely  reporting  of  corrective  action  for  the 
analysis  of  trends  to  prevent  the  development  of  non-conforming  products? 
(MIL-S-52779A  3-2-9) 
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Do  procedures  exist  for  tne  timely  reporting  of  corrective  action  lor  the 
review  of  the  adequacy  of  improvement  of  corrective  measures  to  be  taken,  and 
monitoring  of  tne  implementation  of  determine  effectiveness  of  such  measures? 
(MIL-S-52779A  3-2.9) 

Do  procedures  exist  for  the  timely  reporting  of  corrective  action  for  the 
analysis  or  review  as  otherwise  provided  for  in  the  contract?  (MIL-S-5277'»a 
3-2.9) 


Do  procedures  exist  for  the  monitoring  of  Cats  I  and  II  ECPs? 

Has  the  contractor  established  a  Computer  Program  Library  to  be  used  for  the 
control  of  source  and  object  program  materials?  (MIL-S-52779A  3.2.9) 

Are  the  controls  established  to  control  the  alteration  and/or  changing  of 
source  and  object  program  materials?  (MIL-S-52779A  3.2.5) 

SQA 

It  an  approved  Higher  Order  Language  (HOL),  per  DoDI  5000.31  and  AFR 
300-10, being  used?  If  not,  was  an  economic  analysis  accomplished  f  show  use 
of  approved  HOLs  was  not  cost  effective  or  technically  practical;  has  a  waiver 
been  obtaioed?  (AFR  800-14,  Vol  I,  Para  3e  and  AFSC  Sup  1,  Para  3(1):  Vol  II, 
Para  3-4c) 

Have  minimum  design  character istics,  such  as  spare  memory  and  extra  processing 
capacity,  been  determined?  (AFR  800-14,  Vol  II,  Para  3-4d,  c,  f)  (i.e.,  has  a 
sizing  and  timing  analysis  been  performed?) 

Does  the  SQA  program  provide  for  the  review  and  audit  of  all  software 
documentation?  (MIL-S-52779A  3-2.4) 

Are  design  walk-throughs  required?  (MIL-STD-1679) 

Are  there  existing  standards  and  conventions?  ( MIL-S-52779A  3-2.4) 

Are  standards  and  conventions  being  followed  (as  cited  in  CPDP  and  SOW)? 
(MIL-STD-1679) 

Are  design  problems  identified  and  followed-up  for  completed  corrective  action 
prior  to  approval  of  the  design?  (MIL-S-52779A  3-2-9) 

Are  reviews  of  software  design  accomplished  prior  to  release  for  coding? 
(M1L-S-52779A  3-2.2)  Has  a  CDR  been  held  for  each  CPCI  prior  to  release  for 

coding? 

Does  tne  software  QA  Program  allow  for  scheduled  reviews  and  scheduled  and 
unscheduled  audits?  (MIL-STD-1521A) 

Does  the  SQA  program  provide  for  the  results  of  the  Reviews/Audits  to  be 
documented  and  published?  (MIL-S-52779A  3-9) 
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Does  tne  yA  policy  require  ao  executive  reporting  responsibility  external  to 
the  deve loping/engineer ing/Program  Manager  to  assure  an  objective  evaluation 
of  conformity  ana  progress?  (MIL-S-52779A) 

Do  policies  and  procedures  adequately  cover  program  acceptance?  (MIL-S-52779A ) 

Does  a  policy  exist  regarding  the  use  and  control  of  patches?  ( MIL-STD-1679 ) 

Is  the  use  of  patches  monitored? 

Does  the  SQA  organizaton  have  tne  independence  to  perform  its  tasks? 
(MIL-S-D2779A  3*1) 

Do  tne  personnel  performing  tne  SQA  function  have  sufficient  authority, 
responsibility,  and  freedom  of  action  to  evaluate  software  design  and 
production  activities,  and  to  initiate  and/or  recommend  changes? 

(M1L-S-‘;,2779A  3*2.4) 

Have  the  SQA  procedures  been  accepted  by  tne  QA  (of  tne  Government) 
organization?  (MIL-S-52Y79A  1.2) 

Does  tne  SQA  Plao  identify  any  tools,  techniques  and  methodologies  that  are  to 
be  empLoyed?  (MIL-S-527Y9A  3*2.1) 

Do  these  tools  follow  the  controls  of  the  SQA  Plan?  (MIL-S-52779A  3*2.1) 

Do  the  QA  procedures  provide  for  a  reporting  and  control  system  that  permits 
management  to  monitor  overall  development  status  of  the  software? 

Do  the  QA  procedures  provide  for  a  reporting  and  control  system  that  permits 
decisions  making  on  tne  basis  of  quality  assurance  data? 

Do  the  QA  procedures  provide  for  a  reporting  and  control  system  that  brings 
inadequacies,  discrepancies  and  deficiencies,  as  well  as  proposal  improvements 
to  the  attention  of  appopriate  supervisory  and  management  personnel  in  a 
timely  manner?  (MIL-S-b2779A  3*2*9) 

Does  the  SQA  program  impose  the  requirements  on  all  the  subcontractors? 
(MIL-S-82779A  3*8) 

Does  the  SQA  requirement  conflict  with  other  requirements  of  the  contract? 

Do  the  QA  procedures  provide  for  a  reporting  and  control  system  that  permits 
rapid  and  effective  corrective  action  when  required  with  positive  feedback  and 
response? 

Are  tnere  procurement  control  procedures  and  organization  structure 
requirements  to  assure  that  subcontracted  software  conforms  to  SQA 
requirements  imposed  on  the  prime  contractor? 

Does  tne  QA  program  require  the  cor, duct  of  timely  reviews  and  audits  of  the 
subcontractor ' s  documentation,  procedures  and  program  progress? 


Does  the  QA  program  pr  -vide  tor  formal  inspection  and  acceptance  by  the  prime 
contractor  of  suocontractor  developed  wl.  verables? 
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Does  tne  QA  group  review  all  subcontract  Statements  of  Work  for  inclusion  of 
QA  program  recuirements? 

Do  procedures  exist  for  the  acceptance  of  software  from  a  subcontractor V 

Does  the  SQA  Plan  identify  aod  tools,  techniques,  and  methodologies  that  a r^ 
to  be  employed?  (MIL-S-52779A  3-2.1) 
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CM 

Uo  solicitation  aocuments  incLude  explicit  statements  establishing  Air  Force 
rights  to  CPCIs  and  associated  doucmeotation?  (AFR  800-1A,  Vol  I,  and  AFSC 
3up  l,  Para  3 . i ) 

Does  the  work  breakdown  structure  and  cost  reporting  system  separately 
identify  computer  programs  and  related  tasks?  (AFR  80C-14,  Vol  Ii,  Para  8-3a) 

Are  tne  SqA  procedures  for  auditing  Conf iguration  Management  available  and 
documented?  (MIL-S-52779A  3-27  and  MIL-STD-403) 

Do  QA  procedures  exist  to  review  and  approve  changes  to  configured  software, 
i.e.,  SCP,  ECP ,  etc? 

Do  procedures  exist  fo  the  timely  reporting  of  corrective  action  for  the 
analysis  of  problems  and  deficiency  reports  to  determine  treads  extent  and 
causes?  (MIL-S-52779A  3-2.9) 

Do  procedures  exist  for  tne  timely  reporting  of  corrective  action  for  the 
analysis  of  trends  to  prevent  the  development  of  non-conforming  products? 
(MIL-S-52779A  3-2-9) 

Do  procedures  exist  for  the  timely  reporting  of  corrective  action  for  the 
review  of  the  adequacy  of  improvement  or  corrective  measures  to  be  taken,  and 
monitoring  of  the  implementation  to  determine  effectiveness  of  such  measures? 
(MIL-S-52779A  3-2-9) 

Do  procedures  exist  for  the  timely  reporting  of  corrective  action  for  the 
aoa lysis  or  review  as  otherwise  provided  for  in  the  contract?  (MIL-S-52779 A 
3-2.9) 

Do  QA  procedures  exist  for  the  monitoring  of  the  categorized  failures  i.e., 

Cat  I  &  Cat  II? 

Has  tne  supporting  command  established  a  Computer  Program  Library  to  be  used 
for  the  control  of  source  and  object  program  materials?  (M1L-S-52779A  3-2.5) 

Are  the  controls  established  to  control  the  alteration  and/or  changing  of 
source  and  object  program  materials?  (MIL-S-52779A  3-2.5) 

Testing 

Does  tne  test  program  include  provisions  to  test  computer  equipment  and 
computer  program  configuration  items?  (AFR  800-Hl,  Vol  II,  Para  5-1) 


Are  formal  qualification  test  required  to  verify  CPCi  performance  against  the 
development  specifications?  (AFR  BOO-IH,  Vol  II,  Para  5-3a) 
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Has  the  Preliminary  Qualification  Teat  process  been  evaluated  and  applied? 

(AFR  800-14,  Voi  II,  Para  5-3a(2).(a)) 

Do  provisions  exist  for  integration  and  test  of  CPCIs  into  the  system?  (AFR 
800-14,  Vo l  II,  Para  5-5e) 

Do  the  CPCi  development  specifications  identify  test  approaches  for  qualifying 
the  CPCI  and  contractor  requirements  for  GFE  or  support  requirements?  (AFR 
800-14,  Vol  II,  Para  5-5e) 

Does  the  system  level  DT4E  plan  identify  overall  planning  for  CPCI  testing  and 
does  it  contain  provision  for  installation  and  checkout  of  CPCIs  into 
fol  Low-on  installations?  (AFR  800-14,  Vol  II,  Paras  5-5fd)  4  (2)) 

Are  separate  test  procedures  prepared  for  each  qualification  test  and  do  they 
proper Ly  cover  requirements  for  test  and  evaluation  of  the  CPCIs?  (AFR 
800-14,  Vol  II,  5-5g) 

Are  the  results  of  all  CPCI  tests  documented?  (AFR  800-14,  Vol  II,  Para  5-5h) 

Are  the  procedures  for  the  review  of  test  activities  included  in  your  SQA 
Program?  (MIL-8-52779 A  3-2.8) 

Are  changes  to  CPCI  released  by  a  Time  Compliance  Technical  Order  (TCT0)?  AFR 
800-14,  Vol  II,  Para  6-8d) 

Do  tne  SQA  procedures  provide  for  the  analysis  of  software  requirements  to 
determine  testability?  (MIL-S-52779A  3-2-8) 

Do  SQA  procedures  provide  for  the  review  of  test  plans  for  compliance  with 
appropriate  standards,  performance  requirements  and  other  contractual 
requirements?  (MIL-S-52779A  3-2-8) 

Do  SQA  procedures  provide  for  the  review  of  test  requirement  and  criteria  for 
adequacy,  feasibility  and  completeness?  (MIL-S-52779A  3-2-8) 

Do  SQA  procedures  provide  for  the  review  of  test  procedures  for  compliance 
with  appropriate  standards,  performance  requirements  and  other  contractual 
requirements?  (MIL-S-52779A  3-2-8) 

Do  SQA  procedures  provide  for  the  monitoring  of  tests  and  certification  that 
test  results  are  the  actual  findings  of  the  test?  (MIL-S-52779A  3-2.8) 

Do  SQA  procedures  provide  for  the  review  and  certification  of  test  reports? 
(MIL-S-52779A  3-2.8) 

Do  SQA  procedures  assure  that  test  related  documentation  is  accurately 
maintained  to  allow  repeatibility  of  tests?  (MIL-S-52779A  3-2.8) 
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Documentation 

Has  consideration  been  given  to  deferred  orderiog,  deferred  requisitioning,  or 
accession  list  technique?  Are  the  documentation  requirements  periodically 
evaluated?  (AKK  800-14,  Vol  II,  Para  7-3d) 

Does  the  Data  Management  Officer  track  data  requirements  against  the  CRISP? 

Are  a  acceptance  requirements  for  the  data  defined?  (AFR  800-14,  Vol  II,  para 
7-41 

Have  agreements  from  using  and  supporting  commands  been  obtained  I'or 
modifications  or  deletions  to  data  requirements?  (AFR  800-14,  Vol  II,  Para 
7-4) 

Do  procedures  exist  that  provide  for  in-depth  review  of  the  design 
specification,  interface  specification,  data  base  design  specification  to 
assure  efficiency,  effectiveness,  reliability,  testability  and  adhereoce  to 
standards? 

Do  the  SQA  procedures  provide  for  the  timely  monitoring  of  resources  in  the 
area  of  processing  time,  storage  and  input/output  channels  to  ensure  the 
proper  realization  of  specified  reserves? 

SQA 

li  an  approved  Higher  Order  Language  (HO)  being  used?  (AFR  300-10)  If  not, 
was  an  economic  analysis  accomplished  to  show  use  of  approved  HOLs  was  not 
e03t  effective  or  technically  practical;  has  a  waiver  been  obtained?  (AFR 
800-14,  Vol  I,  Para  3.e  and  AFSC  Sup  1,  para  3.e(l)  (Added);  and  AFR  800-14, 

Vol  II,  Para  3-4c) 

Have  minimum  design  characteristics,  such  as  spare  memory  and  extra  processing 
capacity  (sizing  and  timing  analysis),  been  determined?  (AFR  800-14,  Vol  II, 
Paras  3-4  .d,  e,  and  f) 

Are  de3igo  walk-throughs  required?  (MIL-STD-1679 ) 

Are  standards  and  conventions  being  followed?  (MIL-STD-1679) 

Are  design  problems  identified  and  followed-up  for  completed  corrective  action 
prior  to  approval  of  the  design?  (MIL-S-52779A  3.2.9) 

Has  a  CDR  been  accomplished  prior  to  release  for  coding?  ( MIL-S-52779A  3.2.2) 
Do  policies  and  procedures  adequately  cover  program  acceptance?  (MIL-S-52779A) 
Contractual  Requir erne nt 

Are  appropriate  safeguards  included  in  vendor  contracts  to  support  the 
Cover nraent's  requirements  with  respect  to  system  configuration  level?  (AFR 
800-14,  Vol  II,  Para  8-6) 

Are  all  computer  programs,  equipment,  and  documents  necessary  to  support  and 
modify  the  operational  program  identified  and  deliverable?  (AFR  800-14,  Vol 
II,  Paras  3-8  4  8-3o(3)) 
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Are  the  aelivery  requirements,  including  media  and  form  specified  in  the 
contract?  (AFR  800-14,  Vol  II,  Para  8-5) 

Are  tne  Government's  rights  to  computer  program  and  associated  data  reflected 
in  the  contract  provisions?  (AFR  800-14,  Vol  II,  Para  8-4) 

Does  the  work  breakdown  structure  and  cost  reporting  system  structure  and  cost 
reporting  system  separately  identify  computer  program  and  related  tasks?  (AFR 
800-14,  Vol  II,  Para  8-3a) 

Do  solicitation  documents  include  explicit  statements  establishing  Air  Force 
rights  to  CPCIs  and  associated  documentation?  Are  the  Government's  rights  to 
computer  programs  and  associated  data  reflected  in  the  contract  provisions? 

(AFR  800-14,  Vol  II,  Para  8-4)  (AFR  800-14,  Vol  I  4  AFSC  Sup  1,  Para  3) 

Are  there  procedures  for  design  reviews  to  ensure  that  provisions  of  the 
contract  are  met?  (MIL-S-52779A  3*2-8) 

Do  reviews  and  audits  ensure  that  all  contractual  requirements  are  met? 
(MIL-S-52779A  3*2.8) 

Do  procedures  exist  that  ensure  the  performance  specification  satisfies  the 
requirements  of  the  SOW  and  other  governing  specifications?  (MIL-S-52779A 
3*2.8) 

Do  the  SQA  procedures  assure  the  conduct  of  walk-throughs?  (MIL-STD-1679 ) 

Do  the  procedures  provide  for  the  adherence  to  standards?  (MIL-S-52779A) 

T&T 

Have  using  and  supporting  commands  identified  computer  resources  and 
associated  documentation  necessary  to  support  the  system?  Is  the  CRISP 
continually  updated  to  reflect  the  current  support  concept?  (AFR  800-14,  Vol 
II,  Para  10-3) 

Do  policies  and  procedures  adequately  cover  program  acceptance?  (MIL-S-52779A ) 

Have  system  performance  and  interface  requirements  been  allocated  to  computer 
equipment  and  computer  program  configuration  items?  (AFR  800-14,  Vol  I,  Para 
3*o(2) ) 

Are  periodic  meetings  with  contractor* ,  using  and  supporting  commands  required 
to  assess  tne  progress  of  the  CPCIs?  (AFR  800-14,  Vol  II,  Para  4-10) 

Have  system  studies  beeo  planned/accomplished  to  select  design  approaches  and 
integrate  computers  and  computer  programs  into  the  total  system?  (AFR  800-14, 
Vol  II,  para  4-5a), 
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